WEYL DENOMINATOR IDENTITY FOR FINITE-DIMENSIONAL LIE 

SUPERALGEBRAS 
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Abstract. Weyl denominator identity for the basic simple Lie superalgebras was for- 
mulated by V. Kac and M. Wakimoto and was proven by them for the defect one case. 
In this paper we prove the identity for the rest of the cases. 



0. Introduction 

The basic simple Lie superalgebras are finite-dimensional simple Lie superalgebras, 
which have a reductive even part and admit an even non-degenerate invariant bilinear 
form. These algebras were classified by V. Kac in |K1] and the list (excluding Lie algebra 
case) consists of four series: A{m, n), B{m, n), C{m), D{m, n) and the exceptional algebras 
D(2,l,a),F(4),G(3). 

Let g be a basic simple Lie superalgebra with a fixed triangular decomposition g = 
n_ © f) © n+, and let A+ = A+^q 1J^+,i be the corresponding set of positive roots. The 
Weyl denominator associated to the above data is 



R :-- 



naeA.,(l + e- 



If g is a finite-dimensional simple Lie algebra (i.e., Ai =0), then the Weyl denominator 
is given by the Weyl denominator identity 

ReP = sgn{w)e'"'', 

where p is the half-sum of the positive roots, W is the Weyl group, i.e. the subgroup of 
GL([)*) generated by the reflections with respect to the roots, and sgn(w) G {±1} is the 
sign of w G W. This identity may be viewed as the character of trivial representation of 
the corresponding Lie algebra. 

The Weyl denominator identities for superalgebras were formulated and partially proven 
(for A(m— 1, n— 1), B{m, n), D{m, n) with min(m, n) = 1 and for C(n), -D(2, 1, a), -F(4), G(3)) 
by V. Kac and M. Wakimoto in |KWj . In order to state the Weyl denominator identity for 
basic simple Lie superalgebras we need the following notation. Let A* be the "largest" 
component of Aq, see 12.21 for the definition. Let W be the Weyl group of go, i.e. the 
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subgroup of GL{i)*) generated by the reflections with respect to the even roots Aq, and 
let sgn(w) be the sign of w. One has W = Wi x where be the Weyl group of 
root system A*, i.e. the subgroup of W generated by the reflections with respect to the 
roots from A#. Set po := EaeA+,o"/2> Pi ■= T.aGA+,i «/2> P ■= Po - Pi- A subset 
n of A+ is called a set of simple roots if the elements of 11 are linearly independent and 
A+ C X]«gn^>oQ^- For each functional / the corresponding set A+(/) contains a unique 
system of simple roots, which we denote by n(/). A subset 5* of A is called maximal 
isotropic if the elements of S form a basis of a maximal isotropic space in V. By [KWj . 
A contains a maximal isotropic subset and each maximal isotropic subset is a subset of 
a set of simple roots (for a certain functional /). Fix a maximal isotropic subset S G A, 
choose a set of simple roots 11 containing S, and choose a functional / : — )■ M in such a 
way that 11 = n(/). Let R be the Weyl denominator for this choice of /. The following 
Weyl denominator identity was suggested by V. Kac and M. Wakimoto in |KW] : 



If S is empty (i.e., (— , — ) is a positive /negative definite) the denominator identity takes 
the form Re'' = ^^^yy sgn(w)e"'''. In this case either Ai is empty (i.e., g is a Lie algebra), 
or Q = osp(l,2/) (type B{0,1)). The Weyl denominator identity for the case 05p(l,2/) 
was proven in |K2j : for the case when S has the cardinality one the identity was proven 



The Weyl denominator identity for root system of Lie (super) algebra g can be again 
naturally interpreted as the character of one- dimensional representation of g. The proofs 
in abovementioned cases ( [K2j . [KW] ) are based on a analysis of the highest weights of 
irreducible subquotients of the Verma module M (0) over g. In this paper we give a proof 
of the Weyl denominator identity ([T]) for the case when S has the cardinality greater than 
one. A similar proof works for the case when the cardinality of S is one. Unfortunately, 
our proof does not use representation theory, but requires an analysis of the roots systems. 
The proof is based on a case-by-case verification of the following facts: 

(i) the monomials appearing in the right-hand side of ([1]) are of the form e''~'^, where 
z/ G Q'^ := J2a€u'^>o^ ^^^^ coefficient of is one; 

(ii) the right-hand side of ([T]) is l^-skew-invariant (i.e., w G W acts by the multiplication 
by sgn(w)). 

Taking into account that for any X G V the stabilizer of A in ly is either trivial or 
contains a reflection, and that if the stabilizer is trivial and WX C (po ~ J2a€A+ '^>o(^)y 
then A = Po, we easily deduce the identity ([T]) from (i), (ii). 

I. M. Musson informed us that he has an unpublished proof of the Weyl denominator 
identity for basic simple Lie super algebras. 



(1) 




in [KW] . 
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The Weyl denominator identity for the affinization of a simple finite-dimensional Lie 
superalgebra with non-zero Killing form was also formulated by V. Kac and M. Wakimoto 
in |KW] and was proven for the defect one case. We prove this identity in jG] . 

Acknowledgments . I am very grateful to V. Kac for his patience and useful comments. 
I would like to thank D. Novikov and A. Novikov for their support. 



1. The algebra TZ 

In this section we introduce the algebra TZ. Since the main technical difficulty in 
our proof of the denominator identity comes from the existence of different triangular 
decompositions, we illustrate how the proof works when there is only one triangular 
decomposition (see ll.3fl.4p . 

1.1. Notation. Denote by Q the root lattice of g and by the positive part of root 
lattice := J2a€n ^>ott; we introduce the following partial order on V: 

/i < z/ a (u — n) E ]R>oa. 

Introduce the height function ht : — t- Z>o by 

ht(^m„a) ■=^171^. 

We use the following notation: ior X C M,Y C V we set XY := {xy\x E X,y E ¥}; 
for instance, := Z^qU. 

Note that Aq is the root system of the reductive Lie algebra go. In particular, all 
isotropic roots are odd. Both sets Aq, Ai are ly-stable: WAi = Aj. 

1.2. The algebra TZ. Denote by Qle", u E V] the algebra of polynomials in e", u E V. 
Let TZ be the algebra of rational functions of the form 

X 

naeA+(l + aae-°)-'^' 

where X E Q[e'^ , v E V] and Qa E Q, nia E Z>o. Clearly, TZ contains the rational 
functions of the form — , where X^aa^rria as above. The group W acts on 

TZ by the automorphism mapping to e""^. Say that P E TZ W -invariant (resp., 
W -skew-invariant) if wP = P (resp., wP = sgn{w)P) for every w E W . 
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1.2.1. For a sum Y := bfiC'^, 6^ G Q introduce the support of Y by the formula 

supp(r) := {/i| b, ^ 0}. 
Any element of TZ can be uniquely expanded in the form 

Em s 

For F G 7^ denote by supp(F) the support of its expansion; by above, supp(F) lies in a 
finite union of cones of the form v — . 



1.2.2. We call A G F regular if StabvK A = {id}; we call the orbit W\ regular if A is 
regular (so the orbit consists of regular points). 

It is well-known that for A G the stabilizer Stabvi/ A is either trivial or contains a 
reflection (see Lemma [4.1.11 (ii)). Therefore the stabilizer of a non-regular point A G 
contains a reflection. As a result, the space of VT-skew-invariant elements of Qfe'^, u & V] 
is spanned by 'Y^^iYSgn.{w)e'^'^, where A G is regular. In particular, the support of a 
Vl^-skew- invariant element of Q[e'^, z/ G V] is a union of regular ly-orbits. 



1.3. Lie algebra case. The denominator identity for Lie algebra is 



There are several proofs of this identity. The proof which we are going to generalize is 
the following. Observe that Re'' E Q[e'^, u E V] and supp(-Re'') C (p — Q'^)- Moreover, 
Re'' = riaGA i^"^"^ ~ e~"/^) is VT-skew- invariant, so supp(-Re'') is a union of regular 
orbits lying in (p — Q^). However, Wp is the only regular orbit lying entirely in (p — Q^), 
see Lemma l4.1.1l (iii). Hence Re'' is proportional to "^^^1^ sgn{w)e'^'' . Since the coefficient 
of e'' in the expression Re'' is 1, the coefficient of proportionality is 1. 



1.4. Case Q{n). For the case g := Q{n) one has Qq = 0[(n), W = Sn, Ao+ = A+^i 
{ei - ej}i<i<j<n and so po = pi, p = 0. The Weyl denominator is i? = HaGA+.o TT^ 
For each S C A4. i define 

A{S) := {w G Sn\ wS C A+,o}, a{S) := ^ sgn(w). 

weA(S) 

1.4.1. Proposition. For each S C A+^i one has 

a{S)R= sgn{w)^ 

^65„ 11/3GSU- 



+ e-^P)' 
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Proof. Observe that the support of both sides of the formula hes in — and that the 
coefficients of 1 = e° in both sides are equal to a{S). Multiplying both sides of the formula 
by the ly-invariant expression naeA+ + e~°')e^^ = Y[aeA+ + e~°/^) we obtain 

Since UaeA+o^^ ~ e~°)e^'' = na6A+o(^"^^ ^ ^~"^^) W^-skew-invariant, Y is also W- 
skew-invariant. Clearly, Y G Q[e'^, i' E V], so, by ll.2.2[ supp(F) is a union of regular 
M^-orbits. By above, supp(F) C (po ~ Q^) \ {Po}- However, by Lemma 14.1.11 (iii), any 
regular W^-orbit intersects po + Q^- Hence y = as required. □ 

1.4.2. In order to obtain a formula for the Weyl denominator R, we choose S such that 
0(5") 7^ 0. Taking S = {ei — En, £2 — £n-i, ■ ■ ■ , ^i^] — ^n+i-[f ]} and using Lemma fl.4.3[ we 
obtain the following formula 

which appears in jKW] (7.1) (up to a constant factor). 

Note that such 5* has a minimal cardinality: if the cardinality of 5* is less than [|], 
then a{S) = 0. Indeed, if the cardinality of S is less than [^], then there is a root 
Ei — Ej, which does not belong to the span of S, and thus Se._e^.Vr(S') = W{S); since 
sgn(w) + sgn(se._e^.ty) = 0, this forces a{S) = 0. 

1.4.3. Lemma. Set A := {a e Sn\ < ^ a{i) > a{n + 1 - i)} . Then 

^sgn(a) = K2]! 



Proof. For each a G A let -P(cr) be the set of pairs {{a{l), cr{n); ((t(2), a{n — 1)); . . .}, that 

is P{a) := a (n + 1 - j))}f2v Let B := {a e A\ P{a) = {(j, j + 1)}%}. Define an 

involution / on the set A\B as follows: for a G A \ i? set /(cr) := (i, i + 1) o o", where i is 
minimal such (z, i + ^ P{<^)- Since sgn(cr) + sgn(/(cr)) = 0, we get J2aeA\B — 0- 

One readily sees that B has [n/2]\ elements and that sgn(cr) = 1 for each a E B. Hence 
'^creA^S^i^) ~ as required. □ 

2. Notation 

Let = 00 © 01 be a a basic Lie superalgebra with a fixed triangular decomposition of 
the even part: 0o = n_^o © J) © n+,o- For A{m — l,n — l)-type we put = 0l(m|n) (one 
readily sees that the denominator identities for gl{m\n) imply the denominator identities 
for sl{m\n),psl{n\n)). Let Aq (resp., Ai) be the set of even (resp., odd) roots of 0. Set 
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V = l)^ {so V = span A for q ^ A{m,n) and V = M span A © M for g = A{m,n)). 
Denote by (— , — ) a non-degenerate symmetric bilinear form on V, induced by a non- 
degenerate invariant bilinear form on g. Retain notation of Section [0] and define the Weyl 
denominator. 

The dimension of a maximal isotropic space in V = i)^ is called the defect of g. If g 
is a Lie algebra or g = osp(l,2/) (type B{0,1)) then the defect of g is zero; the defect 
of A[m — 1, n — 1), B{m, n), D{m, n) is equal to min(m, n); for C(n) and the exceptional 
Lie superalgebras the defect is equal to one. Notice that the cardinality of a maximal 
isotropic set S is equal to defect of g. 

2.1. Admissible pairs. The set of positive even roots A_|_ o is determined by the trian- 
gular decomposition go = n_,o © f) © tt+,o (i-e., A+_o is the set of weights of u+,o)- Recall 
that for each maximal isotropic set S there exists a set of simple roots containing S. We 
call a pair {S, U) admissible if C n is a maximal isotropic set of roots and n is a set of 
simple roots such that the corresponding set of positive even roots coincides with A+ o- 

{S, n) is admissible if ^ C H & A+(n) n Aq = A+,o. 
For a fixed set of simple roots n we retain notation of ll.ll and ll.2[ 

2.2. The set A*. Let Ai, A2 be two finite irreducible root systems; we say that Ai is 
"larger" than A2 if either the rank of Ai is greater than the rank of A2, or the ranks are 
equal and Ai C A2. 

If the defect of g is greater than one, then the root system Aq is a disjoint union of two 
irreducible root systems. We denote by A* the irreducible component, which is not the 
smallest one, i.e. Aq = A*]JA2, where A* is not smaller, than A2, see the following 
table: 



A 


A(m-1 


n-1) 




B(m,n) 




D(m,n) 




m > n 


m < n 


m > n 


m < n 


171 = 11 


m> 11 m <n 


A* 




An-l 






Bm OI' Cm 


D C 



The notion of A* in |KW] coincides with the above one, except for the case B{m,m), 
where we allow both choices Bm and Cm, whereas in |KW] A* is of the type Cm- 
Notice that go = Si X S2, where Si,52 are reductive Lie algebras and A*, Aq \ A* are 
roots systems of 5i,52 respectively. We normalize (— , — ) in such a way that A* := {a G 
Aol (a, a) > 0}; then Aq \ A# = {a G Ao| (a, a) < 0}. 

3. Outline of the proof 
3.L Let g be one of the Lie superalgebras A{m — l,n — 1), B{m, n), D{m, n),m,n > 0. 
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3.2. Expansion of the right-hand side of (1). Let (S*, H) be an admissible pair. Set 

(2) X := Y: ^gn(^)^( f ) 

and rewrite the denominator identity ([1]) as Re^ = X. 
Expanding X we obtain 

(3) X = Y.^^^* E,«,o5 sgn(^)(-l)^*'^e^(-)-IHM+-P, 
where 

^{w) := ^ G -Q+ 

/3e5:i«/3<0 

and \w\ is a linear map Z>oS' — defined on /3 G 5 by the formula 



\w\l3 



wf3 for w/3 > 0, 
—w(3 for < 0. 



3.3. Main steps. The proof has the following steps: 

(i) We introduce certain operations on the admissible pairs {S, U) and show that these 
operations preserve the expressions X and Re^. Consider the equivalence relation on the 
set of admissible pairs {S, U) generated by these operations. We will show that there are 
two equivalence classes for D{m,n),m > n and one equivalence class for other cases. 

(ii) We check that supp(X) C (p — Q~^) and that the coefficient of in X is 1 for a 
certain choice of (S, 11) (for D[m,n),m > n we check this for (S, 11) and (S",n), which 
are representatives of the equivalence classes). 

(iii) We show that X is W^-skew-invariant for a certain choice of {S, U) (for D{m, n),m > 
n we show this for (S*, 11) and (5", 11), which are representatives of the equivalence classes). 

For Q of A(n — l,n — 1) type we change (ii) to (ii'): 

(ii') We check, for a certain choice of (5,11), that supp(X) C (p — Q^) and that for 
^ := Y^jB^sP coefficients of e''~'^^ in X and in Re^ are equal for each s G Z>o. 

The choices of (S", II) in (ii), (iii) are the same only for A{m,n) case. 



3.4. Why (i)— (iii) imply (1). Let us show that (i)-(iii) imply the denominator identity 
X = Re^. Indeed, assume that X — Re^ ^ 0. 

Since WS C Ai, X — Re^ is a rational function with the denominator of the form 
n^gA+(l + e"^); we write 

Y YeP^ 
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where Y e Q[e'' , u e V]. One has 

and the latter expression is VT-skew- invariant, since its enumerator is ly-skew- invariant 
and its denominator is ly-invariant. Combining (i) and (iii), we obtain that X — Re'' 
is M^- skew- invariant. Thus Ye''^ is a VT-skew- invariant element of Q[e'^, z/ G V] and so 
supp(ye^^) is a union of regular orbits. 

Observe that supp(i?e^) C {p — Q^) and that the coefficient of in Re^ is 1. Using 
(i), (ii) we get supp(X — Re'') C (p — Q^) \ {p}. Note that the sets of maximal elements 
in supp(F) and in supp(X — Re'') coincide. Thus suppY C (p — Q^) \ {p} that is 
supp(ye''^) C (po ~ Q^) \ {po}- Hence supp(ye''^) is a union of regular orbits lying in 
(Po-Q+)\{po}. 

By 14. 2[ for q ^ 0[(n|n), the set (po — Q^) \ {po} does not contain regular VT-orbit, a 
contradiction. 

Let = %\{n\n). Choose 11 as in 16.31 By 14.21 the regular orbits in (po — Q^) are of the 
form Vr(po — sO with s G Z>o, ^ = X]/3es ^- '^^^ ~ ^ "^Po — Po? so po — sC, is 
the maximal element in its H^-orbit. Thus a maximal element in supp Ye''^ is of the form 
Po — s^, so a maximal element in supp Y is p — sC,. Then, by above, p — sC^ G supp(X — i?e''), 
which contradicts to (ii'). 

4. Regular orbits 

4.1. Let g be a reductive finite-dimensional Lie algebra, let W be its Weyl group, let II 
be its set of simple roots and let 11"^ be the set of simple coroots. For p, defined as above, 
one has (p, a^) = 1 for each a G II. Set 

Qq = ^Q«, q5:=5^q>o«. 

As above, we define partial order on by the formula p < u if (z/ — /i) G Ylia&/\+ IR>o'^- 

Let P C [)^ be the weight lattice of g, i.e. z/ G P iff {u, a^) G Z for any a G II, and 
let P+ be the positive part of P, i.e. v G iff (z/, o^) G Z>o for any a G II. One has 
PcQq. 

4.1.1. Lemma. (i) P = WP+. 

(ii) For any A G t)^ the stabilizer of X in W is either trivial or contains a reflection. 

(iii) A regular orbit in P intersects with the set p + Q^. 

Proof. The group W is finite. Take A G f)}^ and let A' = wX be a maximal element in the 
orbit WX. Since A' is maximal, (A', a"^) > for each a G II. 
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For A G P one has A' G P, so (A', a"^) G Z>o for each a G 11, that is A' G P^, hence (i). 

For (ii) note that, if (A', a'^) = for some a G 11, then Sa G Stabvi/ A', so s^-ia ^ 
Stably A. Assume that (A', a"^) > for all a G 11. Take y E W, y ^ id and write y = y'sa 
for a G n, where the length of y' E W is less than the length of y. Then y'a G A+ (see, 
for instance, [J], A. 1.1). One has y'X' - y'saX' = {\',a^){y'a) > so y'X' > y's^X'. Now 
(ii) follows by the induction on the length of y. 

For (iii) assume that A' is regular, that is (A', a^) > for all a G II. Since A' G P, one 
has (A', a^) G Z>i so (A — p, a"^) > for all a G II. Write A — p = X]/3en -^/S/^- ^ vector 
(l//3)/3en write y > if y/3 > for each (3. The condition (A — p,a^) > for all a G II 
means that Ax > 0, where x = (a;^)/3en and A = {{a'^ , f3))a,i3£n is the Cartan matrix of 
g. From |K3] . 4.3 it follows that Ax > forces x > 0. Hence A — p G Qq as required. □ 

4.2. Now let g be a basic simple Lie superalgebra, Q be its root lattice and be the 
positive part of Q. 

4.2.1. Corollary. Let q be a basic simple Lie superalgebra and g 7^ C{n), A{m,n) . A 
regular orbit in the root lattice Q intersects with the set po + Q>oA4._o- 

Proof. For g C{n), A(m, n) one has QAq = QA and the g-root lattice Q is a subset of 
weight lattice of go. Thus the assertion follows from Corollary 14.1.11 □ 

4.2.2. Case g = Q{{m\n). For g[(m|ra) one has 



Choose a set of simple roots as in 16.31 
Lemma. Let g = Q[{m\n). 

(i) For m ^ n, Wpo is the only regular orbit lying entirely in (po — Q^)- 

(ii) For m = n the regular orbits lying entirely in (po — Q^) are of the form W{pq — 
s^),s> 0, where ^ = " "^i ■ 

Proof. Let l : QA — )• QAq be the projection along ^ (i.e., Kei l = Q^). Since ^ is W- 
invaraint, wt{X) = l{wX). Let WX C (po — Q~^) be a regular orbit. Since l{Q) lies in 
the weight lattice of go, iiWX) intersects with po + Q>oA+^o, by Corollarv 14.1.11 Thus 
WX contains a point of the form po + z/ + g^, where u G Q>o A+ and g G Q. By above, 
^ + q^e -Q+ so G -Q>oA+. 

Consider the case m > n. In this case, for any p G Q>oA_|_ one has (p, ei) ■ (p, Em) < 0. 
Since (^,^1) = {C,£m) 7^ 0, the inclusion G — Q>oA+ implies g = 0. Then u G Q>oA+^o 
and u = u + q^ G —Q'^ so u = 0. Therefore WX = Wpo. Hence Wpo is the only regular 
orbit lying entirely in po — . Since gl(m|n) = gl(n|m), this establishes (i). 



QA = QAq © Q^, where ^ : 




n 
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Consider the case m = n. Set (3i := Si — 6i, /3l := 6i — Si^i. One has ^ = J27=i Pi ^"^^ 
n = {A}r=i 'J {A'}r=i^- The simple roots of A+^q are Si — ^j+i = A + and 5j — = 
P[ + A+i. Thus z/ G Q>oA+ takes the form u = Er=i'^i(A + AO + Ci(A' + A+i) with 
bi, c, G Q>o. By above, u' :=-(// + G Q+. Therefore z/ + z/' G Q ^"^^ A and i/' G 
One readily sees that this implies 6j = Cj = that is z/ = 0. One has fl = Z>o^. 
Hence a regular orbit in po — Q"*" intersects with the set po — ^>o^ as required. □ 

4.2.3. Case C{n). Take 11 = {^i — £2, £^2 — £^3, • • • , — 5i,£^n + ^i}- One has QA = 
QAq © Q5i. We claim that W pQ is the only regular orbit lying entirely in (po — Q^)- 

Indeed, take a regular orbit lying entirely in (po — <5^)- Combining the fact that W5i = 
Si and Corollary 14.1. 11 we see that this orbit contains a point of the form po + u+qSi, where 
^ £ Q>o^o and g G Q. Since — (z/ + q6i) G Q~^, one has q6i G — Q>oQ^- However, 26i is 
the difference of two simple roots {2Si = (e^ + 61) — (e„ — 61)) so Q(5i fl (— Q>o<5^) = {0} 
that is g = 0. The conditions z/ G Q>oAg , — (z/ + g5i) G give z/ = 0. The claim follows. 

5. Step (i) 

Consider the following operations with the admissible pairs (5*, H). First type operations 
are the odd reflections (5, H) 1— )■ (s/^S*, S/jH) = {S \ {/3} U {— s^sH) with respect to an 
element of (3 E S (see 15. 2p . By 15.2^ these odd reflections preserve the expressions X,Re^. 
Second type operations are the operations (5*, H) h-)- (S", H) described in Lemma 15. 3^ 
where it is shown that these operations also preserve the expressions X, Re''. Consider 
the equivalence relation on the set of admissible pairs {S, H) generated by these operations. 
In 15.41 we will show that there are two equivalence classes for D{m,n),m > n and one 
equivalence class for other cases. In 15.51 we will show that if {S, H), {S, H') are admissible 
pairs, then H = H' (on the other hand, there are admissible pairs (S*, H), (S", H) with 
5* 7^ S', see Lemma I5l3|) . 

5.1. Notation. Let us introduce the following operator F : TZ TZ 

F{Y) := sgn{w)wY. 

w€W# 

Clearly, F{wY) = sgn{w)F{Y) for w G VT*, so F{Y) = if wY = Y for some w G W* 
with sgn(w) = — 1. 

For an admissible pair {S, H) introduce 

pp(n) 

Y{S, H) := X{S, H) := F(r(5, H)), 

where p(H) is the element p defined for given H. Note that X = X{S,Il) for the corre- 
sponding pair (S', H). 
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5.2. Odd reflections. Recall a notion of odd reflections, see [S]. Let 11 be a set of simple 
roots and A+(n) be the corresponding set of positive roots. Fix a simple isotropic root 
/3 G n and set 

.,(A+):=A+(n)\{/3}U{-/3}. 
For each P C 11 set S/3(P) := {s/3(a)| a G P}, where 

{—a if a = /3, 
a if (a, /3) = 0, a 7^ /5 

a + (5 if(a,/3)^0. 

By ■s^(A+) is a set of positive roots (i.e., s^(A+) = A+(/) for some functional /) 
and the corresponding set of simple roots is spijl). Clearly, p(s^(n)) = p(n) + /3. 

Let (5", n) be an admissible pair. Take /3 G 5. Then for any /?' G 5 \ {/?} one has 
s^(/3') = /?' so 3^(5") = (5' \ {/?}) U {— /?}. Clearly, the pair {s p{S) , s pijl)) is admissible. 
Since p{sp{Ii)) = p(Jl) + (3, one has Y{S, U) = vls^iS), sp(U)). 



5.3. Lemma. Assume that are isotropic roots such that 

-fES, yen, 7 + 7' G A#, (7', /3) = /or eac/i /3 G 5 \ {7}. 
Then the pair {S' ,11) , where S' := (S'U{7'})\{7}, is admissible and X{S,ir) =X(S",n). 

Proof. It is clear that the pair (S", n) is admissible. Set a := 7 + 7' and let Sa G IV* be 
the reflection with respect to the root a. Our assumptions imply that 

(4) SaP = p] Sai = -7; Sa(3 = (3 foT (3 E S \ {7}. 

Therefore Sa{Y{S',U)) = Y{S,U)e-^ that is F{Y{S',U)) = F{s^{Y{S,U)e-^)) . Since 
F o Sa = —F, the required formula X{S, LI) = X{S', U) is equivalent to the equality 
F(Y{S, n)(l + e"''')) = 0, which follows from the fact that the expression 



F(^,n)(l + e- 



n 



/3e5\{7} 



1 + 6-/3) 



is, by 



), So-invariant. 



□ 



5.4. Equivalence classes. We consider the types A{m — l,n — l), B{m, n), D{m, n) with 
all possible m, n. We express the roots in terms of linear functions 6, . . . , ^m+n, see [Klj . 
such that 



A 


A(m-l,n-l) 


B(m,n) 


D(m,n) 


A+,0 


U 


u' u {mu u {2e.}r=r+i 


u' u {2^,}r=r+i 


Ai 


{±(6 — ^j)}l<i<m<j<m+n 







and 



U := {C,i — C,j\ ^ < i < j ^ 1TL or m + 1 < i < j < m + n}, 
U' := {^i ± C,j\ ^ < i < j ^ rn or m + 1 < i < j < m + n} . 
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5.4.1. For a system of simple roots 11 C spanj^j take an element /n G span{^*}^"|"' 
such that (/n, a) = 1 for each a G 11 (the existence of /n follows from linear independence 
of the elements in 11). Note that /n is unique if g is not of the type A{m — l,n — 1); 
for A^rn — l,n — 1) we fix /n by the additional condition mini<j<m+n(/n, ^i) = 1- In the 
notation of Sect. [Ulone has 11 = n(/n). We will use the following properties of fu'- 

(i) (/n, a) eZ\ {0} for all a G A; 

(ii) if a G A+, then (/n, a) > 1; 

(iii) a root a G A is simple iff {fu, a) = 1. 

Write fu = ^^'^ ^i^i- By (i); the XjS are pairwise different; by (ii) , Xj ± Xj+i > for 
i ^ m [1 < i < m + n — 1); hj (iii), a root C,i — C,j is simple iff Xj — = 1. 

5.4.2. For type A{m — 1, — 1) all roots are of the form — ^j. In particular, {xj}^"^"" = 
{1, . . . ,m + n}. 

Consider the case B{m,n). In this case for each i one has G A+ so Xj > 1. There- 
fore, by (iii), a simple root can not be of the form ±(^j + ^j) so II = {^j^ — ^1^,^12 — 
63, • • • , - and {xi}™t" = {1, . . . , m + n}. 

5.4.3. Consider the case D{m,n). Using (i), (ii) and the fact that 2^rn+n G A+o; we 
conclude 

Vj 2xj G Z; Vz ^ j Xi±Xj eZ\ {0}; 
Xi — Xj > j ~ i foT i < j < m or m < i < j] 
Xj > for j > m; xj + Xj > for i < j < m. 

Recall that, if —{Cp + ■Cg) ^ n l~l Ai, then II' := s__^p_^^(n) contains + C,g. Let us 
assume that C,p + G II fl Ai,p < q, that is Xp + Xg = 1, p < m < g. If p < m, 
then Xp + Xq > ±(xm + Xg) (because Xp ± x^, 2xq > 0), so Xp + Xg > |xm + > 1, a 
contradiction. Hence p = m, in particular, 

(5) ±i^p + Q eUnA,, p<q ^ p = m. 

Since 2xq G Z>o, Xp + Xg = 1 and Xg 7^ Xm, one has Xm < 0. Therefore the assumption 
implies 

Xm + Xg = 1 Sz q > m Sz Xm < 0. 
Since ^m-i — G A+ there exists a simple root of the form — ■Cm (s 7^ m). 

First, consider the case when — im G II, that is Xg XfYi 1. SillCG Xyyi < 0, one 

has Xs + Xm < Xs — Xm = i so Xs + Xm < 0, that is —{C,s + C,m) G A_|_. Therefore s > m 
and Xg + Xs = 2, because x^ + x^ = a;^ — x^ = 1. Since Xm+n-i > 1/2 + i for i < n, we 
conclude that either Xg = 3/2, x^ = — l/2,Xs = 1/2, which contradicts to x^ + a;^ 7^ 0, or 
q = s = m + n, x^+n = ^,Xm = 0. Hence — G H implies q = m + n, x^+n = 1, = 

and ^rn+n ± G H. 
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Now consider the case when — — 6m ^ H. Then s > m and Xm + Xq = —Xm — Xg = 1 
so Xq — Xs = 2. Since Xq — Xq+i > i one has s = q + l or s = q + 2. Ifs = g + 2, 
then we have 2 = Xq — Xq+2 = {xq — x^+i) + (x^+i — that is Xq — Xq+i = 1 so 

Xm + Xq+i = 0, a contradiction. Hence s = q + 1 that is —Xm — Xg+i = 1. For i < m one 
has < Xj + = — a^g+i — 1 + Xj so 1 < Xj — Xg+i. Therefore for i < m, t > g + 1 one has 
Xi + Xt > Xj — Xt > 1, so the roots ±6, ± are not simple. Assume that m>n and (S", 11) 
is an admissible pair. Then for each m < t < m + n there exists it < m such that one of 
the roots ±C,t ± C,it is simple (and lies in S) and 7^ for t ^ p. By above, Zg+i = m and 
there is no suitable for t > g + 1. Hence q + 1 = m + n and (— — C,m+n) G 'S', that is 

5.4.4. We conclude that if (5*, H) is an admissible pair for q = D{m,n), then one of the 
following possibilities hold: either all elements of S are of the form — ^j, or all elements 
of S except one are of the form C,i — and this exceptional one is (3, where 

(1) /3 = C,m + ^m+n ^nd ^m+n ~ ^ I^i 

{2) m < n and /3 = + 6s, ""^ < s < m + n, — 6s+i — 6m ^ 

(3) /3 := —(6m + 65)5 the pair {spS, spU) is one of whose described in (l)-(2). 

5.4.5. Consider the case D{m,n), n > m. Then A* = {6» ± 6i; 26i}IL'L\i- Let (S, H) 
be an admissible pair. If 6m + 6s ^ 5" for s < m + n, then, by above, —(6m + 6s+i) ^ 
H and the pair (5", H) is equivalent to the pair {{S \ {6m + 6s}) U {—6m — 6s+i},n), 
which is equivalent to the pair {{S \ {6m + 6s}) U {6m + 6s+i}; ■S-^^-^^.^^H). Thus a 
pair {S, H) with 6m + 6s ^ "S" is equivalent to a pair (5", H') with 6m + 6m+n ^ "S". If 
6m + 6m+n ^ "S", then, by above, 6m+n — 6m ^ n and the pair {S, H) is equivalent to the 
pair ((5* \ {6m+n + 6m}) U {6m+n — 6m}, H). We couclude that any pair {S, H) is equivalent 
to a pair {S', H'), where 5' = {6i - 6iJ^i- 

Consider the case D{m, n), m > n. Then A* = {6i =t6j}i^i- By above, any pair {S, H) 
is equivalent either to a pair (S", H'), where 5" = {6j — 6j }?=!; to a pair (S", H'), where 

S' = {6i - 6i, }r=l' U {6m+„ + 6m} and 6m+„ - 6m G H'. 

5.4.6. Let {S, H) be an admissible pair. We conclude that any pair {S, H) is equivalent 
to a pair (S",H'), where either S' = {6i — 6ji}ii'i*'™'"'*, or, for D{m,n), m > n S' = 

{6i ~ 6«j }r=l U {6m+n + 6m} and 6m+n ^ 6m ^ H'- 

5.5. Fix a set of simple roots of g and construct /n as in 15.41 We mark the points Xj on 
the real line by a's and 6's in one of the following ways: 

(M) mark Xj by a (resp., by 6) if 1 < i < m (resp., if m < i < m + n); 

(N) mark Xj by b (resp., by a) if 1 < z < m (resp., if m < i < m + n). 
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We use the marking (M) if A* lies in the span of and the marking (N) if A* 

hes in the span of Note that in all cases the number of as is not smaller than 

the number of 6s. 

We fix A* and an admissible pair (S', 11) such that S = {^i — or for 

D{m, n),m> n, S = {^i- jr=i^ U {U+n + U}- 

If C,i — C,j E S (resp., + G S) we draw a bow (resp., ^) between the points Xi 
and Xj. Observe that the points connected by a bow are neighbours and they are marked 
by different letters (a and b). We say that a marked point is a vertex if it is a vertex of a 
bow. Note that the bows do not have common vertices and that all points marked by b 
are vertices. 

From now one we consider the diagrams, which are sequence of as and 6s endowed with 
the bows (we do not care about the values of Xi). For example, for q = A{4, 1) and 11 = 

{6-6; 6-6; 6-6; 6-6; 6-6; 6-6} we choose / = a+2a+36*+46+5a+66*+76* 

and taking 5* = {6 — 6; 6 — 6} we obtain the diagram aaa bb aa; for q = B{2, 2) 
and n = {6 - 6; 6 - 6; 6 - 6; 6} we choose / = 6* + 26* + 36 + 46* and taking 

5 = {6 — 6; 6 — 6} we obtain the diagram a ba b for the marking (N), and 

6 a6 a for the marking (M). Observe that a diagram containing ^ appear only in 
the case D{m, n), m > n, and such a diagram starts from a b, because ^m+n + (,m ^ S 
forces ^m+n — £ n, which implies Xm = 0, Xm+n = l,Xi > for all i ^ m. 

5.5.1. Let us see how the odd reflections and the operations of the second type, intro- 
duced in Lemma 15. 3[ change our diagrams. 

Recall that 6 ~ 6 ^ ^+ ^« ^ -^i- -^^^ '-"^'^ simple root P one has S/3(A+) = 
(A+ \ {13}) U {—/?}, so the order of XjS for S5p_5^(A+) is obtained from the order of XjS 
for A+ by the interchange of Xp and Xq (if 6 — 6 ^ LI fl Ai). Therefore the odd reflection 
with respect to 6 — 6 ^ corresponds to the following operation with the diagram: we 
interchange the vertices (i.e., the marks a,b) of the corresponding bow: 

...a ^ b... H- ...b a...; ...b ^ a... ...a ^ b... 

If the diagram has a part a ba (resp. ab a), where the last (resp. the first) 
sign a is not a vertex, and the corresponding points, then the quadrapole 

(5*, n), 7 := 6 — 6; 7' 6 — 6 (resp. 7' := 6 — 6' 7 6 — 6) satisfies the assumptions 
of Lemma 1531 The operation (S", LI) ^ (S", LI), where S' := {S \ {7}) U {7'} corresponds 
to the following operation with our diagram: a ba ^ ab a (resp. ab a ba). 

Hence we can perform the operation of the second type if a, 6, a are neighbouring points, 
b is connected by ^ with one of as and another a is not a vertex; in this case, we remove 
the bow and connect b with another a: 

...ab ^ a... (— )■ ...a ba...] ...a ba... ^ ...ab a.... 
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Since both our operations are involutions, we can consider the orbit of a given diagram 
with respect to the action of the group generated by these operations. Let us show that all 
diagrams without ^ lie in the same orbit. Indeed, using the operations a ^ 6 i— )■ 6 ^ a 
and ah a \^ a ha we put h to the first place so our new diagram starts from h a. 
Then we do the same with the rest of the diagram and so on. Finally, we obtain the 
diagram of the form h ah a . . .h a . . . a. By the same argument, all the diagram 
starting from 6 a lie in the same orbit. Hence, for g ^ D{m, n),m > n all diagrams lie 
in the same orbit, and for g = D{m,n),m > n there are two orbits: the diagrams with 
^ and the diagrams without 

5.5.2. We claim that if {S, 11), {S, 11') are admissible pairs, then 11 = 11'. It is clear that 
if the claim is valid for some S, then it is valid for all S' such that {S, U) is equivalent to 
(S", n'). Thus it is enough to verify the claim for any representative of the orbit. Each 5* 
determines the diagram and the diagram determines the order of Xj's (for instance, the 
diagram h ah a . . .h aa . . . a gives Xm+n < Xm < Xm+n-i < Xm-i < ■ ■ ■ < Xm+i < 

Xm-n+l < Xm-n < ■ ■ ■ < Xi foT the marking (M) and Xm < Xm+n < Xm-l < Xm+n-l < 

. . . < Xi < Xn+1 < Xn < . . . < Xm+1 foY the marking (N)). For A{m — l,n — 1), B{m,n) 
one has {xj}™"^" = {1, . . . , m + n}, by 15.4. 2^ so each diagram determines 11. 

For _D(m, n), m > n the above diagram means that —C,m+n + ^m, —^m + ^m+n-i, ... G 11. 
Since 2^m+n ^ "we conclude that 2^rn+n ^ n that is Xm+n = ^ and {xj}™"^" = 
. . . ,m + n — |}. Hence the above diagram determines H. 

For D{m,n),m < n the same reasoning shows that the diagram a hh a . . .h 
aa . . .a determines H. 

It remains to consider the case D{m,n), m > n and the diagram a ^ hh ah ^ 
a . . .h ^ aa . . .a. In this case, + (,m+n ^ S so, by above, = and {xj}™"^" = 
{0, . . . , m + n — 1}. Thus the diagram determines H. 

5.5.3. Consider the case g 7^ D{m,n),m > n. Fix an admissible pair (5, H). By 15. 5^ 
any admissible pair (5", H') is equivalent to an admissible pair (5, H"); by 15.5.21 one has 
H = H" so (5', H') is equivalent to {S, H). 

Consider the case g = D{m,n),m > n. Fix admissible pairs (5*, H), (S",H'), where S 
consists of the roots of the form — and S' contains + im+n- Arguing as above, we 
conclude that any admissible pair (S"', H") is equivalent either to (5, H) or to (5",H'). 

5.6. Corollary. (i) If (3,11), (5, H') are admissible pairs, then U = U' . 

(a) For g 7^ D{m,n),m > n there is one equivalence class of the pairs (S*, H) and the 
left-hand (resp., right-hand) side of ^ is the same for all admissible pairs (5*, H). 

(Hi) For Q = D{m,n),m > n there are two equivalence classes of the pairs (S", H). In 
the first class S consists of the elements of the form ±(,^j — ^j) and in the second class S 
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contains a unique element of the form ±(^j + The left-hand side (resp., right-hand) 
of (dj is the same for all admissible pairs {S, U) belonging to the same class. 

6. Steps (ii), (ii') 

6.1. Assume that 

. . Va G n {a, a) > 0; 

^ ^ W* is generated by the set of reflections {sa\ {a, a) > 0}. 

We start from the following lemma. 

6.1.1. Lemma. One has 

(i) p > wp for all w G W^*; 

(ii) the stabilizer of p in is generated by the set {sa\ (a, a) > 0, (a,p) = 0}. 

Proof. Since = \{a,a) > for all a G II, one has > for all /3 G A^. Take 

w G VT*, w 7^ id. One has w = w'sjb, where w' G W*, (3 G A+ are such that the length of 
w is greater than the length of w' and w' (3 G A+ (see, for example, [J], A.l). One has 

p-wp = p-w p+ J^-^ • P)- 

Now (i) follows by induction on the length of w, since (p, (3) > 0, (3) > 0, w'f3 G A_|_. 
For (ii), note that p > w'p by (i), thus wp = p forces p = w'p and (p, /3) = 0. Hence (ii) 
also follows by induction on the length of w. □ 

6.1.2. Retain notation of For w e W* one has —ip{w), \ w\p G Q^, by definition, 
and wp < p, by Lemma [6.1.11 Therefore (f{w) — \w\p + wp < p and the equality means 
that p = 0, wp = p and fi^w) = 0, that is wS C A+. The above inequality gives 
supp(X) G {p — Q'^)- Moreover, by above, the coefficient of e'' in the expansion of X is 
equal to 

^ sgn(M;). 

6.2. Root systems. Recall that we consider all choices of A_|_ with a fixed A_|_ q. From 
now on we assume that m > n (we consider the types A{m, n), B{m, n), B{n, m), D{m, n), 
D{n,m)) and we embed our root systems in the standard lattices spanned by {Si, Sj : 1 < 
i < m,l < j < n} chosen in such a way that A* = Aq fl spanjej}^]^. More precisely, for 
A{m, n),m > n we take 

A+,0 = {£i - Sj - Sj'\l <i<i'<m,l<j<j'< n}, Ai = {±{ei - Sj)}; 

for other cases we put 

U' := {si ± Si'] 5j ± 5j'\l < i < i' < m, 1 < j < j' < n} 
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and then 

A+,0 = t/' U {ei}Zi U {2Sj}]=i, Ai = {±ei ± 5j, ±5j} for B{m, n),m>n 

and B(n, n), A* = B{ 
A+,0 = U'U {2ei}Zi U Ai = {±ei ± 5,-, ±£,} for B{n, m),m>n 



n] 



A+,0 = U'U {25,}^=i, 
A+,o = t/'U{2£,}-i, 



Ai = {±£,±5,} 

Ai = {±£,±5,} 



and _B(n, n), A* = C(n); 
for D(m, n),m > n; 
for Z^(n, m),m > n. 



We normalize the form (— , — ) by the condition = —{Si,Sj) = 6ij. One has 

A# = Ao n span{£:i}^i = {a G Ao| (a, a) > 0}. 

6.3. Choice of (S*, 11). For the case B{n,n) set 

S := {6i - Ei}, n := {6i - ei,ei- 62, ^2 - £2, ■ ■ ■ , - ^n, ^n}, 
(the root En niay be even or odd, depending on the choice of A*). For other cases set 

In order to describe 11 introduce 
and set 



A 


A(m-l,n-l) 


B(m,n), B(n,m), n > m 


D(n,m), m > n 


D(m,n) m > n 


n 


P 




P U {2e^} 


P U {em-l + Sm] 



n := {ei - 5i,5i - 62,62 - 52,52 - £3, - 5n,6n + 5„} for D{n,n). 

Note that the assumptions (jSD hold. 



6.4. Step (ii): the case (Sm + ^n) ^ H. By l6.1.2[ supp(X) C {p — Q^) and the coefficient 
of e'' in the expansion of X is '^^izy/#-wp=p «)5ca+ sgn(w). Thus it is enough to show that 

(7) w G s.t. wS C A+, = p =^ w = id. 

6.4.1. Since (a, a) > for each a G 11, one has (p, /3) = for /3 G A iff /3 is a linear 
combination of isotropic simple roots. 

Consider the case q 7^ D{n,n) (one has p = for D{n,n)). In this case (p, /3) = for 
/3 G A* forces /3 = 6i — 6j for i < j < min(m, n + 1). From Lemma [6.1.11 we conclude 



(8) 



iS if TJl/ — 77, 

for 7^ /^(n, n) Stab^# p = <{ . 

^ ' ^ ^ I Sn+i, if m > 



where S'fc C VT* is the symmetric group, consisting of the permutations of £1, . . . , (that 
is for w E Sk one has W6i = 6j- and ji = i for i > k). 
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6.4.2. Take w G Stabvi/# P such that wS C A+. Let us show that w = id. 

Consider the case when q ^ B{n,n), D{n,n). Then S = {si — Combining ([8]) 

and the fact that ej — 6i E A+ iff j < i, we conclude that wEi = sj. for ji < i if i < n and 
ji = i for i > n + 1. Hence w = id as required. 

For g = D{n, n) one has S = {Si — Since —Si — 6j ^ for all the condition 

wS C A_|_ forces w & Sn (see 16.4. 1] for notation). Repeating the above argument, we 
obtain w = id. 

For Q = B{n, n) one has 5* = {5i — ei}"^^^. Combining ^ and the fact that 5i — Sj G A+ 
iff j > i, we obtain for alH = 1, . . . , n that wei = ej^ for some ji < i. Hence w = id as 
required. This establishes ([7]) and (ii) for the case (e^ + ^n) ^ H. 



6.5. Step (ii): the case (em+Sn) G H. Consider the case /^(m, n), m > n,{em+Sn) G H. 
We retain notation of 16. 21 and choose a new pair {S, H): S = {si — 5i}i=i U {Sm + Sn} and 

n := {Ei — 5i, 5i — €2, ■ ■ ■ , 5n-l — £n} U {Si — U {^m-l ^ <^n., — ^m, + £m}- 

The assumptions ([6]) are satisfied. By I6.1.2[ the coefficient of in the expansion of 
X is equal to J2weA^S^i'^)y where A := {w G Stabiy* p| wS C A+}. Let us show 
that A = {id, Se^_j_e„, S£„^_-^_e^Se^_-^+£^}; this implies that the coefficient of in the 
expansion of X is equal to 1. 

Take w G W^* such that wS C A+ . Note that —ej — 6i ^ A+ for all i,j and for i < n 
one has Sj — SiE A+ iff j < z. The assumption wS C A_|_ means that wEi — SiE A_|_ for all 
i < n and wEm + G A+. For i < n this gives wEi = Ej^ for some ji < i. Hence wEi = Ei 
for i = l,...,r;, — 1. The remaining condition wEm + 5„ G A_|_ means that wEm = Sj,^ or 

For m = n + 1 one has p = 0, so w G A iff w G W*, wS C A_,_. Thus, by above, w E A 
iff WEi = Ei for i < n = m — 1 and WEm G {±£^5 ^m-i}- 

Take m > n + L The roots (3 G A* such that (p, /3) = are of the form 13 = Ei — Ej for 
i<j<n or (3 = Em-i ±£^m- From Lemma lB.l.ll we conclude that the subgroup Stabp^* p 
is a product of Sn defined in 16.4.11 and the group generated by the reflections 
By above, w E A iS wEi = Ei for i < m — 1 and WEm G {±Em, Em-i}- 

Since ly* is the Weyl group of D{m), i.e. the group of signed permutations of {Ei}^i, 
changing the even number of signs, the set 

{w G WEi = for i < m — 1 & WE-m G {±Em,£m-l}} 

is {id, S£^_-^_£„^, Se^_i_e^Se^_j+e^}. HeUCC A = {id, Se^_j^_Smy ^Sm-i-em^Sm-i+em} 

quired. This establishes (ii) for the case {Em + 5„) G H. 
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6.6. Step (ii'). Consider the case g[(n|n). One has p = 0. Set ^ := J2i3£sf^ = 
^^j. Let us verify that 

(9) n supp(i?e'' - X) = 0. 

Indeed, it is easy see that ^ has a unique presentation as a positive hnear combination of 
positive roots: 

(10) ^ = rriaa, riia > =^ mp = 1 for /3 G 5, nia = for a ^ S". 

This imphes that for s ^ Z>o the coefficients of e"**^ in Re'' = R and in X are equal to 
zero, and that the coefficient of e"**^ in R is equal to (—1)*" for s G Z>o. It remains to 
show that the coefficient of e~*^ in X is (—1)*" = (— 1)^*'^'*^^ It is enough to verify that 
\w\fi — ifiw) = sC, for w G implies w = id. Assume that \w\fi — (p{w) = s^. By 
definition, \w\fi, —ip{w) G Q^. From (jTOjl we conclude that \w\^,—ip{w) G Z>oS. Recall 
that ip{w) = X]/3GSto/3GA_ ""^Z^- ( |T0|) . —fiw) G Z>oS' implies {—wf3) G S for each 
/3 E S such that w/3 G A_. However, {—wf3) ^ S for any w G VT*, (3 E S, because 
—w{ei — 6i) = Si — wEi ^ S. Thus wS C A+ that is wSi = Si. for < i. Hence w = id. 
This establishes ([9]). 



7. ly- IN VARIANCE: STEP (ill) 



In this section we prove that X defined by the formula is H^- skew- invariant for a 
certain admissible pair [S, H); for the case D{m, n) m > nwe prove this for two admissible 
pairs (5", H) and (S",H'), which are representatives of the equivalence classes defined in 
Sect. El 

Recall that X is iy#-skew- invariant and that A = A# ]J A2, that is 1^ = W* x W2. 

7.1. Operator F. Recall the operator F : TZ ^ TZ given by the formula F{Y) := 
E^GW/# sgn(w)wF. Clearly, w{F{Y)) = F{wY) for w e W2 and F{wY) = w{F(Y)) = 
sgn{w)F{Y) for w G ly*. In particular, F{Y) = if wY = Y for some w G PF* with 
sgn(w) = — 1. One has 

X = F(Y), where Y :-- 



Suppose that B E 71 is such that W2W1B = B for some Wi G W*,W2 G W2, where 
sgn(wiW2) = 1. Then 



^F{B) = F{w^^B) = F{wiB) = sgn{wi)F{B), 



that is W2F{B) = sga{w2)F{B). 

As a result, in order to verify PF-skew-invariance of F{B) for an arbitrary B eTZ, it is 
enough to show that for each generator y of W2 there exists z G such that sgn{yz) = 1 
and yzB = B (we consider y running through a set of generators of W2). 



20 



MARIA GORELIK 



7.2. Root systems. We retain notation of 16. 21 for Ao,+ and Ai, but, except for A{m — 
1, n — 1) we do not choose the same pairs {S, U) as in 16.31 

For A{m — l,n — l) we choose S, U as in 16.31 (S := {si — 6i}). For other cases we choose 

S '■= {Sn-i ~ ^m-i}i=o Slid 

U := P U {Sm} ioi B{m^n),B{n,m) 

n := P U {Em + ^n} for D{m, n) m > n, 
n := P U {26m}, for D{n, m) m > n, 

where P := {Ei —62, ■ ■ ■ , Em-n-l ~ ^m-n, ^m-n ~ ^m-n+l; ^m-n+l ~ ^2, ■ ■ ■ , ~ ^m} 

for m > n, and P := {6i — 61,61 — 62, ■■■ ,Sn — En} for m = n. For D{m, n),m > n case we 
consider two admissible pairs: (S, 11) and (S", 11), where 5" := {5n-i — £m~i}^=i^{5n+Em}- 

Recall that {si, Sj) = —{Si, 6j) = 6ij and notice that (a, a) > for all a G 11. 

7.3. 5'„-invariance. Let Sn C W2 be the group of permutations of 5i, . . . , 5„. In all cases 
5" is of the form S = ±{5j — Er+i} for r = or r = m — n. For i = 1, . . . ,n — 1 one has 
{p,6i-6i+i) = {p,er+i-er+i+i) = 0. Therefore the reflections Se^^^_er+^+l: ^6-5,+^ stabilize 
p. Since Se^+,-e,+,+^ss^-5,+-, stabilizes the elements of S, one has Ss,.+i-er+,+iSs^-s^^^Y = Y 
for i = 1, ... ,n — 1. Using mi we conclude that X is S'„-skew-invariant. In particular, 
this establishes VT-invariance of X for A(m, n)-case. 

For D{m,n),m > n case consider the admissible pair (5", 11). Arguing as above, one 
sees that ss^-5^^-^{X) = —X for i = 1, . . . , n—2. Since 6n-i—£m-i, ^n+^m £ S' the product 
w := Se„„i+e^S5„_i_5„ stabilizes the elements of 5". Since {p,5i) = {p,6m^n+i) = for 
i = 1, . . . ,n, w stabilizes p. Thus wY = Y and so = —X, by 17.11 Hence X is 

^^-skew-invariant . 

7.4. B{m,n), 3(71,771), D{m,n),rn > 71 cases. In this case W2 is the group of signed 
permutations of so it is generated by ss^ and the elements of 5"^. In the light 
of 17.11 and 17. 3[ it is enough to verify that ss^SeJY = Y. Set 

(3 := 5n - £m ^ S. 

Consider the cases P(m, n), P(n, m). In this case PF* is the group of signed permuta- 
tions of {^i}™ 1. Since jS, Em e 11, one has (p, Em) = (p, 5n) = | so ss„Se^p = p+P. Clearly, 
ss„Ss^ stabilizes the elements of \ {/3}, and ss„Se^f3 = —/3. As a result, ss^s^^Y = Y 
as required. 

Consider the case D(m,n),m > 71. In this case VT* is the group of signed permu- 
tations of {£i}, which change even number of signs. Notice that Ss^_„Ss^ G VT* and 
sgn{ss^_„SsJ = 1. Set w := Se„,_„Se„S5„. One has (p, (5„) = (p,e„^) = (p,£m-„) = so 
wp = p. Since w stabilizes the elements of S" \ {/?}, one has wY = e~^Y . We obtain 

ss^F{Y) = ssMse^.^Se^Y) = F {s^^s^^^s^JY) = F{wY) = Fie-^Y) 
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and so 



(1 + ssjF{Y) = F((l + e-^)Y) = F{- ' = 0, 

li/3'es\{^}V-'- -r e ^ J 



where the last equahty follows from the fact that the reflection S£^_„-e^ stabilizes p and 
S \ {/?}. Hence (1 + ss„)F(Y) = as required. 

For the admissible pair (S", 11) we obtain the required formula (1 + ss„)F[Y) — along 
the same lines substituting (3 by 6n + £m- 

7.5. Case D{n, m),m > n. In this case is the group of signed permutations of {si}, 
and W2 is the group of signed permutations of which change even number of signs. 

Note that the reflection ss^ does not lie in W2, but s^.A = A, so s^. acts on TZ and this 
action commutes with the operator F. Since W2 is generated by ss^s^j and the elements 
of Sn, it is enough to verify that ssiSs2F{Y) — F{Y). Set /3i :— 6i — Sm-n+i € S. One has 
(p, Em-n+i) = (p, ^i) = 1 SO S5^Se^+n-ip ^ p + 2/3i that is sSiSe^+n-iY = e^'Y . Therefore 

(1 - ss^)F{Y) = F{Y) + F{ss^s,^^n-^Y) = + eP^)Y) = F{^ 77^7^) = 0' 

because s^^+n-i ^ stabilizes p+A and the elements of S'\{/3j}. Thus S5.F(y) = F{Y) 
so 55.55. F(y) = -F(i^) for any i, j. Hence X = F{Y) is H^2-skew-invariant. 
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